Spreadsheet applications such as, for example, Microsoft Excel®, are widely used in many fields and are increasingly important for analyzing data in today's business environments. For example, data analysts regularly use client-side (or local) spreadsheet applications as a tool for performing computationally expensive spreadsheet tasks including, but not limited to, consolidating and massaging data, producing charts, performing complex calculations, and the like.
However, as the spreadsheet tasks become increasingly complex, and as the amount of data grows, the calculations become increasing expensive to execute in terms of calculation time, power consumption, and rendering for user observation. For example, client-side spreadsheet applications can be limited in the number or rows (or volume) of data that can be processed. Moreover, the data must be processed synchronously which results in a variety of inefficiencies including exceedingly long wait times, lost calculation data, timeouts, etc., among other inefficiencies.
External Big Data frameworks can be utilized, in conjunction with client-side spreadsheet applications, to process and analyze large datasets. Typically, scripts are developed and executed to create external data pipelines between the external Big Data systems, e.g., Cosmos, Hadoop, SQL, etc. The scripts can, for example, direct the external Big Data systems to aggregate datasets, perform one or more processes on the datasets, and/or move the processed data to a temporary storage location where results can be exported to the client-side spreadsheet application. Unfortunately, there is currently no way to control the external pipelines without writing individualized scripts for each external Big Data system. This process is time consuming, burdensome, and synchronous, e.g., once the process is kicked off by a computer, it must be finished on that computer.
Overall, the examples herein of some prior or related systems and their associated limitations are intended to be illustrative and not exclusive. Upon reading the following, other limitations of existing or prior systems will become apparent to those of skill in the art.